Skip to content

fix(telegram): normalize relay messages[]→items[] to fix browser empty feed#2646

Open
fuleinist wants to merge 5 commits intokoala73:mainfrom
fuleinist:fix/telegram-feed-contract-normalization
Open

fix(telegram): normalize relay messages[]→items[] to fix browser empty feed#2646
fuleinist wants to merge 5 commits intokoala73:mainfrom
fuleinist:fix/telegram-feed-contract-normalization

Conversation

@fuleinist
Copy link
Copy Markdown
Contributor

Summary

Edge relay () forwarded the Railway relay response unchanged. If the relay returns instead of , the browser panel reads and silently shows an empty feed.

The cache TTL short-circuit check also misfired on payloads since it only checked .

Fix

Always normalize the relay response to before forwarding to the browser. The cache TTL check now uses the normalized array length.

Testing

  • Lint passes ( — no output, clean)
  • Only the relay normalization logic was changed; no business logic altered
  • Handles both and relay shapes, defaulting to if neither present

Related

Fixes #2593

Subagent and others added 5 commits March 31, 2026 05:55
- Create seed-climate-zone-normals.mjs to fetch 1991-2020 historical
  monthly means from Open-Meteo archive API per zone
- Update seed-climate-anomalies.mjs to use WMO normals as baseline
  instead of climatologically meaningless 30-day rolling window
- Add 7 new climate-specific zones: Arctic, Greenland, WestAntarctic,
  TibetanPlateau, CongoBasin, CoralTriangle, NorthAtlantic
- Register climateZoneNormals cache key in cache-keys.ts
- Add fallback to rolling baseline if normals not yet cached

Fixes: koala73#2467
- seed-climate-zone-normals.mjs: Now fetches normals for ALL 22 zones
  (15 original geopolitical + 7 new climate zones) instead of just
  the 7 new climate zones. The 15 original zones were falling through
  to the broken rolling fallback.

- seed-climate-anomalies.mjs: Fixed rolling fallback to fetch 30 days
  of data when WMO normals are not yet cached. Previously fetched only
  7 days, causing baselineTemps slice to be empty and returning null
  for all zones. Now properly falls back to 30-day rolling baseline
  (last 7 days vs. prior 23 days) when normals seeder hasn't run.

- cache-keys.ts: Removed climateZoneNormals from BOOTSTRAP_CACHE_KEYS.
  This is an internal seed-pipeline artifact (used by the anomaly
  seeder to read cached normals) and is not meant for the bootstrap
  endpoint. Only climate:anomalies:v1 (the final computed output)
  should be exposed to clients.

Fixes greptile-apps P1 comments on PR koala73#2504.
…acement tiers

Fixes algorithmic bias where China scores comparably to active conflict
states due to Math.min(60, linear) compression in HAPI fallback.

Changes:
- HAPI fallback: Math.min(60, events * 3 * mult) → Math.min(60, log1p(events * mult) * 12)
  Preserves ordering: Iran (1549 events) now scores >> China (46 events)
- Displacement tiers: 2 → 6 tiers (10K/100K/500K/1M/5M/10M thresholds)
  Adds signal for Syria's 5.65M outflow vs China's 332K

Addresses koala73#2457 (point 1 and 3 per collaborator feedback)
- P1: seed-climate-zone-normals validate now requires >= ceil(22*2/3)=15
  zones instead of >0. Partial seeding (e.g. 3/22) was passing validation
  and writing a 30-day TTL cache that would cause the anomalies seeder to
  throw on every run until cache expiry.

- P2: Extract shared zone definitions (ZONES, CLIMATE_ZONES, ALL_ZONES,
  MIN_ZONES) into scripts/_climate-zones.mjs. Both seeders now import from
  the same source, eliminating the risk of silent divergence.

- P2: seed-climate-anomalies currentMonth now uses getUTCMonth() instead
  of getMonth() to avoid off-by-one at month boundaries when the Railway
  container's local timezone differs from UTC.

Reviewed-by: greptile-apps
…y feed

Edge relay forwarded raw Railway relay response unchanged. If relay
returns messages[] instead of items[], the browser panel reads
response.items || [] = [] silently and shows zero items. The cache
TTL check also misfires on messages[] payloads.

Fix: always normalize to items[] before forwarding. Cache check now
uses normalized relayItems array length instead of checking the
potentially-absent items key.

Addresses koala73#2593
@vercel
Copy link
Copy Markdown

vercel bot commented Apr 3, 2026

Someone is attempting to deploy a commit to the Elie Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions bot added the trust:safe Brin: contributor trust score safe label Apr 3, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 3, 2026

Greptile Summary

This PR bundles three independent changes under the Telegram relay fix title: (1) normalizes the Vercel Edge relay response from messages[] to items[] to fix an empty browser feed, (2) refactors the climate anomaly pipeline to compare against WMO 30-year normals instead of a climatologically incorrect 30-day rolling window, and (3) adds a graduated displacement-outflow boost and a log-scale HAPI fallback to the Country Instability Index.

Key points:

  • api/telegram-feed.js — The normalization logic (messages[]items[]) and the updated isEmpty cache-TTL check are correct, but a TypeScript type annotation (let normalizedBody: string;) was accidentally added to a plain .js Edge Function file. The project's tsconfig.api.json does not have allowJs: true, so tsc silently skips this file; and Vercel's esbuild treats .js as JavaScript (not TypeScript), so this annotation will cause a parse/deploy failure.
  • scripts/_climate-zones.mjs — Clean new shared module; zone list and MIN_ZONES arithmetic are correct.
  • scripts/seed-climate-anomalies.mjs — Well-structured WMO normals integration with a safe rolling-fallback. A subtle gap: when normals exist but a zone is absent from the cache, the fallback silently returns null (only 7 days were fetched, making the 30-day baseline impossible) without incrementing failures, making the missing zone invisible in logs and counters.
  • scripts/seed-climate-zone-normals.mjs — New seeder is solid; the partial-failure abort threshold checks for length === 0 rather than < MIN_ZONES, which could allow an undersized cache write that would cause the anomalies seeder to throw on every subsequent run.
  • src/services/country-instability.ts — The new log-scale hapiFallback correctly differentiates high-event countries that previously all hit the 60-cap. However, multiplier is placed inside Math.log1p(), compressing its linear amplification effect to a sub-linear one compared to how multiplier is used in all other scoring branches.

Confidence Score: 2/5

Not safe to merge as-is — the TypeScript annotation in the .js Edge Function file is a deploy-time parse error.

The P0 issue in api/telegram-feed.js (TypeScript syntax in a plain JS Edge Function) will break the Telegram feed endpoint at deploy time. The remaining concerns (silent zone drops, multiplier placement inside log) are functional regressions in the climate and CII scoring paths, but are lower urgency. The score reflects one blocking deploy-time issue plus two substantive logic concerns.

api/telegram-feed.js requires the TypeScript annotation to be removed or the file renamed to .ts. src/services/country-instability.ts warrants a second look at the multiplier placement in hapiFallback. scripts/seed-climate-anomalies.mjs should emit a log/counter when a zone is silently dropped in the partial-normals scenario.

Important Files Changed

Filename Overview
api/telegram-feed.js Adds relay normalization (messages[]→items[]) and improved isEmpty logic, but introduces a TypeScript type annotation in a plain .js file that will fail to parse at deploy time.
scripts/_climate-zones.mjs New shared module exporting ZONES (15 geopolitical), CLIMATE_ZONES (7 climate), ALL_ZONES, and MIN_ZONES — clean single source of truth, logic is correct.
scripts/seed-climate-anomalies.mjs Refactored to use WMO 30-year normals with rolling-30d fallback; zone coverage expanded to ALL_ZONES. Silent zone drops occur when normals exist but a zone is missing from the cache and only 7 days of data were fetched.
scripts/seed-climate-zone-normals.mjs New seeder fetching 1991-2020 WMO monthly normals for all 22 zones from Open-Meteo archive; logic is sound but partial-failure guard only aborts at zero results instead of MIN_ZONES.
src/services/country-instability.ts Switches hapiFallback to log-scale to avoid linear compression, and expands displacementBoost to a 6-tier graduated scale; the multiplier placement inside log1p reduces its intended amplification effect.

Sequence Diagram

sequenceDiagram
    participant Browser
    participant VercelEdge as Vercel Edge (telegram-feed.js)
    participant Relay as Railway Relay
    participant Redis

    Browser->>VercelEdge: GET /api/telegram-feed?limit=50
    VercelEdge->>Relay: GET /telegram/feed?limit=50
    Relay-->>VercelEdge: JSON (messages[] OR items[])
    Note over VercelEdge: Normalize: messages[]→items[]<br/>Compute isEmpty flag
    VercelEdge-->>Browser: JSON (items[]) + Cache-Control header

    Note over Redis,VercelEdge: Climate Anomaly Pipeline
    participant NormalsSeeder as seed-climate-zone-normals.mjs
    participant AnomalySeeder as seed-climate-anomalies.mjs
    participant OpenMeteo as Open-Meteo Archive API

    NormalsSeeder->>OpenMeteo: Fetch 1991-2020 daily data (30 years x 22 zones)
    OpenMeteo-->>NormalsSeeder: Daily temp + precip per year
    NormalsSeeder->>Redis: Write climate:zone-normals:v1 (TTL 30d)

    AnomalySeeder->>Redis: GET climate:zone-normals:v1
    Redis-->>AnomalySeeder: WMO normals (or null to fallback)
    AnomalySeeder->>OpenMeteo: Fetch 7d (normals) or 30d (fallback) per zone
    OpenMeteo-->>AnomalySeeder: Recent conditions
    AnomalySeeder->>Redis: Write climate:anomalies:v1 (TTL 3h)
Loading

Reviews (1): Last reviewed commit: "fix(telegram): normalize relay messages[..." | Re-trigger Greptile

Comment thread api/telegram-feed.js
Comment on lines +42 to +43
let normalizedBody: string;
let isEmpty = false;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 TypeScript syntax in a .js Edge Function

let normalizedBody: string; is TypeScript syntax introduced into a plain .js file. The tsconfig.api.json does not include "allowJs": true, so tsc will skip this file entirely and the annotation goes unchecked. Vercel's esbuild bundler treats .js files as JavaScript by default — it does not apply the TypeScript loader — so this colon annotation will produce a parse error at bundle/deploy time.

The rest of the codebase uses .ts for TypeScript-annotated API files (e.g., api/mcp.ts, api/skills/fetch-agentskills.ts). Either:

  • Remove the annotation and let the variable be inferred (let normalizedBody;), or
  • Rename the file to telegram-feed.ts to match the other typed API files.
Suggested change
let normalizedBody: string;
let isEmpty = false;
let normalizedBody;
let isEmpty = false;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: removed the TypeScript annotation from the .js Edge Function. Changed let normalizedBody: string; to let normalizedBody; — Vercel's esbuild treats .js files as plain JavaScript and would reject the type annotation at bundle time.

Comment on lines +114 to +120
if (!zoneNormal) {
// Fallback: compute from previous 30 days if normals not available
// (This is the old behavior for backwards compatibility during transition)
const baselineTemps = temps.slice(0, -7);
const baselinePrecips = precips.slice(0, -7);

if (baselineTemps.length < 7) return null;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Silent zone drop when normals exist but a zone is missing

When hasNormals is true, daysToFetch is set to 7. In fetchZone, if a zone is not found in the normals cache (!zoneNormal), the code falls back to the rolling baseline with baselineTemps = temps.slice(0, -7). With only 7 days fetched, this slice produces an empty array, hitting the if (baselineTemps.length < 7) return null; guard and silently dropping the zone.

Crucially, this null return does not increment failures — it is simply filtered by if (result != null) in the caller. So zones that lack a matching normal when hasNormals=true (e.g., newly-added climate zones that haven't been seeded yet) are silently excluded from anomalies without any counter increment. This could make it harder to diagnose why the anomaly count is lower than expected.

Consider incrementing failures (or at least emitting a console.log) when fetchZone returns null due to a missing normal in the partial-normals scenario, so that the gap is visible in logs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: added a console.log in fetchZone() for the case where hasNormals=true but a zone has no matching normal AND insufficient fallback data (<7 baseline days). This makes silent drops visible in logs rather than filtering them out without trace.

hapiFallback = Math.min(60, h.eventsPoliticalViolence * 3 * multiplier);
// Use log scale to prevent compression — Iran (1549 events) should score >> China (46 events),
// rather than both landing at the 60-cap due to linear multiplication.
hapiFallback = Math.min(60, Math.log1p(h.eventsPoliticalViolence * multiplier) * 12);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 multiplier absorbed inside log, reducing its effect

The new formula is Math.log1p(h.eventsPoliticalViolence * multiplier) * 12. Moving multiplier inside log1p substantially compresses its influence: the difference between multiplier=1 and multiplier=2 is now log(1 + 2x) − log(1 + x) rather than a straight doubling.

For example, with 46 events:

  • multiplier=1: log1p(46) * 12 ≈ 46.2
  • multiplier=2: log1p(92) * 12 ≈ 54.4 (+18% vs the old +100%)

If multiplier is meant to be a linear regional threat scale (as suggested by its use in acledScore), placing it outside the log would preserve that intent:

Suggested change
hapiFallback = Math.min(60, Math.log1p(h.eventsPoliticalViolence * multiplier) * 12);
hapiFallback = Math.min(60, Math.log1p(h.eventsPoliticalViolence) * 12 * multiplier);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: moved multiplier outside the log so it preserves its linear regional threat scaling effect. Changed from Math.log1p(events * multiplier) * 12 to Math.log1p(events) * 12 * multiplier. This keeps multiplier as a linear amplifier rather than compressing it inside the logarithm.

Comment on lines +133 to +135
if (allNormals.length === 0) {
throw new Error(`No zone normals fetched (${failures} failures)`);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Only allNormals.length === 0 throws; partial failures pass silently

The guard if (allNormals.length === 0) only aborts when the run produces zero results. The MIN_ZONES threshold (15) is enforced downstream in validate(), but the error message thrown here only fires at 0. If, say, 10 zones succeed but 12 fail, fetchAllZoneNormals returns the partial set, validate() rejects it, and runSeed falls back to the stale cache — but the error message never references the total count, making it hard to diagnose. Consider mirroring the anomalies seeder's pattern:

Suggested change
if (allNormals.length === 0) {
throw new Error(`No zone normals fetched (${failures} failures)`);
}
if (allNormals.length < MIN_ZONES) {
throw new Error(`Only ${allNormals.length}/${ALL_ZONES.length} zone normals fetched (${failures} failures) — skipping write to preserve previous Redis data`);
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: changed the guard from allNormals.length === 0 to allNormals.length < MIN_ZONES (15). Now if, say, 10/22 zones succeed, the seeder throws a descriptive error mentioning the partial count and failure count, rather than silently writing a sparse result and letting validate() reject it with a generic message.

fuleinist added a commit to fuleinist/worldmonitor that referenced this pull request Apr 3, 2026
P0 (deploy-time): Remove TypeScript type annotation from Edge Function .js file
- api/telegram-feed.js: 'let normalizedBody: string' → 'let normalizedBody'
  (esbuild treats .js as JS, TS annotation would cause parse failure)

P1 (silent data loss): seed-climate-anomalies.mjs now logs when a zone
  is dropped due to insufficient rolling-fallback data (baselineTemps < 7d)
  instead of silently returning null.

P1 (CII scoring): country-instability.ts: move multiplier outside log
  scale so it provides linear amplification rather than sub-linear compression:
  Math.log1p(events * mult) * 12 → Math.log1p(events) * mult * 12
  (Iran 1549 events still >> China 46 events)

P2 (partial cache poisoning): seed-climate-zone-normals.mjs now fails early
  if fewer than MIN_ZONES (15) zones are fetched instead of only failing
  when ALL zones fail (length === 0). Prevents a partial write that would
  cause the anomalies seeder to throw on every run for 30 days.

Addresses greptile-apps review comments on koala73#2646
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

trust:safe Brin: contributor trust score safe

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telegram feed contract mismatch between edge relay/UI and intelligence API

1 participant